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DETAILED ACTION 

1 . Applicant's response to the last Office Action, filed July 1 1 , 2008 has entered and 
made of record. 

2. In view of the Applicant amendments, the objection to Specification is expressly 
withdrawn. 

3. In view of the Applicant amendments, the rejection of claims 15-16 and 45 under 
35 U.S.C §112 is expressly withdrawn. 

Response to Arguments 

4. Applicant's arguments filed July 1 1 , 2008 have been fully considered but they are 
not persuasive. 

(a) Applicant argues (page 24, lines 1-11) that the Examiner remarked that 
separation of a face into sub-regions corresponds to normalizing a face region with 
respect to a standard size based on separation of eyes, nose, or mouth, or combination 
thereof.... Applicant's use of "separation of is as a noun and refers to a distance 
between things , such as the distance between the nose and mouth and either eye and 
the nose or mouth of a human face. 

However, in response to Applicant's arguments the Examiner disagrees, because 
of the following reason: First the "separating of means "to divide into constituent parts" 
(Merriam- Webster) which clearly different from "distance between things"; second : the 
claim language (see element (d) of claim 1) recites: "wherein the normalized face 
regions comprise spatially normalized face regions, which are normalized with respect 
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to orientation or pose, or both". From the claim language, the separation of a face into 
sub-regions corresponds clearly to normalizing a face region based on separation of 
eyes, nose or mouth, or combination thereof. 

Therefore, the rejection of claim 1 is proper and should be sustained. 

(b) Applicant argues (pages 25-26) that the combination of Morimoto et al. and 
Tony J. does not yield Applicant's invention, nor would such combination produce an 
advantageous product that would work properly without having to take extraneous 
steps, such that one skilled in the art would not be motivated to combine Morimoto et al 
with Tony. J. Furthermore, the Applicant argues (page 26, lines 21-25) that Morimoto et 
al. describe a technique that is designed to be used in a non-analogous field.... 

In response to Applicants arguments, the Examiner disagrees. 

As shown in figure 4, Morimoto et al. disclose one or more processor-readable 
media having stored thereon processor-readable code (column 3, line 20-22) including 
a database of face print data corresponding to detected face regions within images 
acquired (column 1, line 10-11) with an image acquisition device (13 in figure 1, column 
3, line 4) and digitally-stored (21 in figure 2,column 3, line 24-25) wherein the face print 
image data are stored within the media for access by processor (column 3, line 23-25) 
comprises: 

(a) an image data component (extracted means) including acquired digital image 
data including content data and unique identifiers corresponding to individual acquired 
digital images or face regions therein, or both; (column 1 , line 47-49); 
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(b) an identity data component (storage means) including an identification listing 
of known identities to which identified face regions detected within the acquired digital 
image data have been determined to correspond; (column 1, line 45-47); 

(c) a face recognition data component (collation means), comprising for an 
individual known identity (column 1, line 49-54): 

(i) an appearance table (table of Fig. 3) including one or more identity 
entries for the known identity, (column 3, line 59-61), (the table of figure 3, 
comprises an appearance characteristics corresponding to the identities entries); 

(ii) one or more identity tables corresponding to the one or more identity 
entries (given name) in the appearance table (column 3, line 59-60); 

(iii) one or more face class tables corresponding to one or more face class 
entries of the one or more identity tables, where each face class table comprises 
one or more face print image entries corresponding to face prints (column 3, line 
47-48; and column 3, line 56-59). 

However, Morimoto et al. do not teach explicitly the normalizing of face 
regions, comprising spatially normalized face region, which are normalized with respect 
to size, orientation or pose, or luminance normalized face region, or combination 
thereof. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions comprises spatially normalized face 
region, which are normalized with respect to the pose of an individual's face (Page 61, 
line 17-22). 
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All of the claim elements are known in references Morimoto et al. and Tony J. 
The only difference is the combination of Face normalization with the face print image 
data. 

Thus, it would have been obvious to one having ordinary skill in the art to use the 
Face normalization and recognition as though by Tony J. with the face prints image data 
as shown in Morimoto et al, since the Face normalization and recognition could be used 
in combination with the face prints image data to achieve the predictable results of 
permitting of an automatic machine-based face detection and recognition in 
uncontrolled environments (Page 1, chapter 1, line 11-12). 

Therefore, the combination of Morimoto et al. and Tony J. is proper. 

In response to applicant's argument that the Morimoto et al. technique is 
nonanalogous art, it has been held that a prior art reference must either be in the field of 
applicant's endeavor or, if not, then be reasonably pertinent to the particular problem 
with which the applicant was concerned, in order to be relied upon as a basis for 
rejection of the claimed invention. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 
(Fed. Cir. 1992). In this case, Morimoto et al. deals with specifying an identical person 
by collating a face as organic characteristics of the human being, and the Tony J. 
reference deals with the face normalization and recognition, which making the two 
references analogous to each other. 

Therefore, the rejection of claims 1-4, 7-10, 12-13, 15-17, 22-23, 26-30, 33-40, 
43, 45-51 , and 58-60 and their dependent claims is proper and should be sustained. 
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Claim Rejections - 35 USC S 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1-4, 7-10, 12-13, 15-17, 22-23, 26-30, 33-40, 43, 45-51, and 58-60 are 
rejected under 35 U.S.C. 103(a) as being unpatentable over Morimoto et al. (US 
6,418,235) in view of Tony S. Jebara "3D Pose Estimation and Normalization for Face 
Recognition", Department of Electrical Engineering, McGill University, May 1996, 
chapter 4, P-P 61-68. 

(1) Regarding claim 1: 

As shown in figure 4, Morimoto et al. disclose one or more processor-readable 
media having stored thereon processor-readable code (column 3, line 20-22) including 
a database of face print data corresponding to detected face regions within images 
acquired (column 1, line 10-11) with an image acquisition device (13 in figure 1, column 
3, line 4) and digitally-stored (21 in figure 2,column 3, line 24-25) wherein the face print 
image data are stored within the media for access by processor (column 3, line 23-25) 
comprises: 

(a) an image data component (extracted means) including acquired digital image 
data including content data and unique identifiers corresponding to individual acquired 
digital images or face regions therein, or both; (column 1 , line 47-49); 
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(b) an identity data component (storage means) including an identification listing 
of known identities to which identified face regions detected within the acquired digital 
image data have been determined to correspond; (column 1, line 45-47); 

(c) a face recognition data component (collation means), comprising for an 
individual known identity (column 1, line 49-54): 

(i) an appearance table (table of Fig. 3) including one or more identity 
entries for the known identity, (column 3, line 59-61), (the table of figure 3, 
comprises an appearance characteristics corresponding to the identities entries); 

(ii) one or more identity tables corresponding to the one or more identity 
entries (given name) in the appearance table (column 3, line 59-60); 

(iii) one or more face class tables corresponding to one or more face class 
entries of the one or more identity tables, where each face class table comprises 
one or more face print image entries corresponding to face prints (column 3, line 
47-48; and column 3, line 56-59). 

However, Morimoto et al. do not teach explicitly the normalizing of face 
regions, comprising spatially normalized face region, which are normalized with respect 
to size, orientation or pose, or luminance normalized face region, or combination 
thereof. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions comprises spatially normalized face 
region, which are normalized with respect to the pose of an individual's face (Page 61, 
line 17-22). 
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It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where normalizing face regions, is to achieve this goal. Therefore, it would 
have been obvious to one having ordinary skill in the art at the time of the invention, to 
apply the Tony's teaching of face normalization and recognition, with the Morimoto et al. 
system, because such combination, would permit automatic machine-based face 
detection and recognition in uncontrolled environments (Page 1, chapter 1, line 11-12). 

(2) Regarding claim 17: 

As shown in figure 4, Morimoto et al. disclose one or more processor-readable 
media having stored thereon processor-readable code (column 3, line 20-22) including 
a database of face print data corresponding to detected face regions within images 
acquired (column 1, line 10-11) with an image acquisition device (13 in figure 1, column 
3, line 4) and digitally-stored (21 in figure 2,column 3, line 24-25) wherein the face print 
image data are stored within the media for access by processor (column 3, line 23-25) 
comprises: 

(a) an image data component (extracted mean) including acquired digital image, 
or a pointer to the location of said image (column 1, line 47-49), and additional data 
(attribute data) associated with said image (column 1, line 48), including content data 
(face data) (column 3, line 50-52), and unique identifiers (personal data) (column 3, line 
44-46), (the personal data is read as unique identifier, such as a given name of person 
to be registered and registration item) corresponding to the acquired digital images or 
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face regions therein, or both, and wherein the image data component further comprises 
an image list of the acquired digital image data (attribute data) (column 4, line 1 3-1 7); 

(b) an identity data component (storage means) including an identification listing 
of known identities to which identified face regions detected within the acquired digital 
image data have been determined to correspond (column 1, line 45-47); 

(c) a face recognition data component (collation means), comprising for an 
individual known identity (column 1, line 49-54): 

(i) one or more identity tables (table of figure 3) corresponding to one or 
more identity entries (column 3, line 3, line 48-50), (the table of figure 3 has the 
identity characteristics corresponding to the identity entries). 

(ii) one or more face class tables (table of figure 3) corresponding to one 
or more face class entries of the one or more identity tables, wherein each face 
class table comprises one or more faceprint entries (Column 3, line 56-67), (the 
table of figure 3 has the face class table corresponding to face class entries of 
the identity table). 

However, Morimoto et al. do not teach explicitly the normalizing of face regions, 
comprising spatially normalized face region, which are normalized with respect to size, 
orientation or pose, or luminance normalized face region, or combination thereof. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions comprises spatially normalized face 
region, which are normalized with respect to the pose of an individual's face (Page 61, 



Application/Control Number: 10/764,335 Page 10 

Art Unit: 2624 

line 17-22), size (average 3D face) (Fig. 4.7, Page 65, line 9-13), and luminance 
normalized face region (page 61, line 9-10). 

It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where normalizing face with respect to poses of an individual's face, is to 
achieve this goal. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time of the invention, to apply the Tony's teaching of normalizing face with 
respect to pose of an individual's face, with the Morimoto et al. system, because such 
combination, would permit automatic machine-based face detection and recognition in 
uncontrolled environments (Page 1, chapter 1, line 11-12). 

(3) Regarding claim 34: 

As shown in figure 4, Morimoto et al. disclose one or more processor-readable 
media having stored thereon processor-readable code (column 3, line 20-22) including 
a database of face print data corresponding to detected face regions within images 
acquired (column 1, line 10-11) with an image acquisition device (13 in figure 1, column 
3, line 4) and digitally-stored (21 in figure 2,column 3, line 24-25) wherein the face print 
image data are stored within the media for access by processor (column 3, line 23-25) 
comprises: 

(a) an image data component (extracted means ) including acquired digital image 
data including content data and unique identifiers corresponding to individual acquired 
digital images or face regions therein, or both; (column 1 , line 47-49); 
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(b) an identity data component (storage means) including an identification listing 
of known identities to which identified face regions detected within the acquired digital 
image data have been determined to correspond (column 1, line 45-47); 

(c) a face recognition data component (collation means), comprising for an 
individual known identity (column 1, line 49-54): 

(i) one or more identity tables (table of figure 3) corresponding to one or 
more identities, wherein each identity table comprises one or more face class 
entries each defined by values of one or more face classifier parameters (figure 
3, column 3, line 44-46), (the table of figure 3 as the combination of identity table 
and face class entries) 

(ii) one or more face class tables (table of figure 3 ) corresponding to the 
one or more face class entries of the one or more identity tables, wherein each 
face class table comprises one or more faceprint image entries corresponding to 
faceprints from the acquired digital image data. (Column 3, line 50-52), (the table 
of figure 3 comprises face class table which comprises face print image entries). 

However Morimoto et al. do not teach explicitly the normalizing of face regions, 
comprising spatially normalized face region, which are normalized with respect to size, 
orientation or pose, or luminance normalized face region, or combination thereof. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions comprises spatially normalized face 
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region, which are normalized with respect to the pose of an individual's face (Page 61, 
line 17-22), size (average 3D face) (Fig. 4.7, Page 65, line 9-13), and luminance 
normalized face region (page 61, line 9-10). 

It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where normalizing face with respect to poses of an individual's face, is to 
achieve this goal. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time of the invention, to apply the Tony's teaching of normalizing face with 
respect to pose of an individual's face, with the Morimoto et al. system, because such 
combination, would permit automatic machine-based face detection and recognition in 
uncontrolled environments (Page 1, chapter 1, line 11-12). 

(4) Regarding claims 2 and 35: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the image data component further comprises an image list of the 
acquired digital image data (attribute data) (column 4, line 13-17). 

(5) Regarding claims 3 and 22: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where at least one group of image data comprises a face region list 
including one or more entries each corresponding to an identified face candidate region 
within an acquired digital image (column 3, line 28-30). 



Application/Control Number: 10/764,335 Page 13 

Art Unit: 2624 

(6) Regarding claims 4 and 23: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the face region list further including one or more links, 
corresponding to the one or more entries, to one or more known identities (personal 
identification) within the identification listing of the identity data component (column 4, 
line 49-53), (the known identity within the identification listing is read as a personal 
identification such address). 

(7) Regarding claims 7. 26. and 36: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the known identities correspond to handles identifying a known 
person (column 4, line 18-26), (the body type data, sexuality data, and age group data 
are read as a known identities stored in the storage, which are corresponding to the 
registered person or known person). 

(8) Regarding claims 8. 27, and 37: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the identity data component further comprises database links to 
face recognition data of the face recognition component (figure 3, column 3, line 50-55), 
(it is read that the identity data comprises links to face recognition data in table of figure 
3). 

(9) Regarding claims 9. 28. and 38: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the identity data component further comprises one or more 
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database links to personal data (given name of the registered person) associated with 
one or more known identities (figure 3, column 3, line 48-50), (the known identity is read 
as the given name of the registered person and has a link to personal data in table of 
figure 3). 

(10) Regarding claims 10. 29. and 39: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the identity data component comprises a table of personal data 
associated with a known identity (see table of figure 3), (the personal data is read as an 
attribute data which is associated with known identity). 

(11) Regarding claims 12 and 33: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where each identity table comprises one or more face class entries (face 
data) each defined by values of one or more face classifier parameters (see table of 
figure 3), (the face data is read as face class entries which is define by one or more of 
contour line, and an area). 

(12) Regarding claims 13 and 43: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where at least two identity entries are characterized separately due to at 
least one distinguishing appearance characteristic (figure 4, step n3, column 4, line 15- 
16), (in step n4, it is read that the attribute data includes the body type data, which is 
interpreted as one of the distinguishing appearance that can make difference between 
two identities entries). 
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(13) Regarding claims 15 and 45: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the appearance table comprises a list of links to one or more 
identity tables associated with distinct appearances (body type) determined for the 
known identity (figure 3, column 3, line 50-55), (the distinct appearance is read as the 
body type which is associated to the identity of the person to be registered). 

(14) Regarding claims 16 and 46: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the one or more identity tables further comprise one or more links 
corresponding to the one or more face class tables (figure 3, column 3, line 46-48), (it is 
read that the one or more identity data in the table such as given name or ID, has a link 
to the face data in the table such as eyes, and position of nose). 

(15) Regarding claims 30 and 40: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the personal data comprises full name, one or more address, one 
or more phone numbers, one or more email address, one or more web address, or 
combination thereof (table of figure 3, column 3, line 44-45 and line 48-50). 

(16) Regarding claim 47: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the one or more face class tables (table of figure 3) comprises one 
or more of the previously determined value range of the one or more face classifier 
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parameters (Figure 3, column 3, line 44-46), (the table of figure 3 comprises a face 
class table which includes a value range of face parameter). 

(17) Regarding claim 48: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where each value range is uniquely associated with the identified and 
user confirmed face region detected within the acquired digital image (column 1 , line 47- 
49), (the value range in the table of figure 3 is associated with the identified face region 
detected within the acquired digital image). 

(18) Regarding claims 49, 50, and 51: 

Morimoto et al. disclose one or more processor-readable media (column 3, line 
20-22) as described in claims 1,17, and 34 above. 

However, Morimoto et al. do not explicitly mention the system, where the 
normalized face regions are normalized prior to extracting face classifier parameters 
therefrom. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions are normalized prior to extracting face 
classifier parameters therefrom (Page 80, line 15-16), (Tony J. is clearly normalizing the 
face before the recognition and classification of the identity on the basis of the 
variance). 

It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where the normalized face regions are normalized prior to extracting face 
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classifier parameters, is to achieve this goal. Therefore, it would have been obvious to 
one having ordinary skill in the art at the time of the invention, to apply the Tony's 
teaching of normalizing face regions prior to extracting face classifier parameters, with 
the Morimoto et al. system, because such combination, would permit automatic 
machine-based face detection and recognition in uncontrolled environments (Page 1, 
chapter 1, line 11-12). 

(19) Regarding claims 58. 59. and 60: 

Morimoto et al. disclose one or more processor-readable media (column 3, line 
20-22) as described in claims 1,17, and 34 above. 

However, Morimoto et al. do not teach explicitly the system, where the 
normalized face regions are normalized with respect to pose, and then with respect to 
orientation, and then with respect to size. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions are normalized with respect to pose 
(Page 61, line 17-19) and then with respect to orientation (Page 61, line 18-19), and 
then with respect to size (segmenting and histogram of face) (Page 61, line 20-23), (it is 
read the segmenting and histogram of face comprise the size of face) 

It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where the normalized face regions are normalized with respect to pose, than 
orientation than size, is to achieve this goal. Therefore, it would have been obvious to 
one having ordinary skill in the art at the time of the invention, to apply the Tony's 
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teaching where normalizing with respect to pose, than orientation than size, with the 
Morimoto et al. system, because such combination, would permit automatic machine- 
based face detection and recognition in uncontrolled environments (Page 1, chapter 1, 
line 11-12). 

7. Claims 5-6 and 24-25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Morimoto et al. (US 6,418,235) in view of Tony S. Jebara "3D Pose Estimation and 
Normalization for Face Recognition", Department of Electrical Engineering, McGill 
University, May 1996, chapter 4, P-P 61-68, and Watanabe (US-PGPUB 
2003/0048926). 

(1) Regarding claims 5 and 24: 

As shown in figure 4, Morimoto et al. disclose one or more processor-readable 
media having stored thereon processor-readable code (column 3, line 20-22) including 
a database of face print data corresponding to detected face regions within images 
acquired (column 1, line 10-11) with an image acquisition device (13 in figure 1, column 
3, line 4) and digitally-stored (21 in figure 2,column 3, line 24-25) wherein the face print 
image data are stored within the media for access by processor (column 3, line 23-25) 
comprises: 

(a) an image data component (extracted means) including acquired digital image 
data including content data and unique identifiers corresponding to individual acquired 
digital images or face regions therein, or both; (column 1 , line 47-49); 
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(b) an identity data component (storage means) including an identification listing 
of known identities to which identified face regions detected within the acquired digital 
image data have been determined to correspond; (column 1, line 45-47); 

(c) a face recognition data component (collation means), comprising for an 
individual known identity (column 1, line 49-54): 

(i) an appearance table (table of Fig. 3) including one or more identity 
entries for the known identity, (column 3, line 59-61), (the table of figure 3, 
comprises an appearance characteristics corresponding to the identities entries); 

(ii) one or more identity tables corresponding to the one or more identity 
entries (given name) in the appearance table (column 3, line 59-60); 

(iii) one or more face class tables corresponding to one or more face class 
entries of the one or more identity tables, where each face class table comprises 
one or more face print image entries corresponding to face prints (column 3, line 
47-48; and column 3, line 56-59). 

However, Morimoto et al. do not teach explicitly the following items: 

1) the normalizing of face regions, comprising spatially normalized face region, 
which are normalized with respect to size, orientation or pose, or luminance normalized 
face region, or combination thereof. 

2) the multiple tables of image classification events, occasions, locations, or 
place, or other categories to which groups of multiple images of image data are 
determined to belong. 
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(a) Obviousness in view of Tony J. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions comprises spatially normalized face 
region, which are normalized with respect to the pose of an individual's face (Page 61, 
line 17-22), size (average 3D face) (Fig. 4.7, Page 65, line 9-13), and luminance 
normalized face region (page 61 , line 9-10). 

It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where normalizing face with respect to poses of an individual's face, is to 
achieve this goal. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time of the invention, to apply the Tony's teaching of normalizing face with 
respect to pose of an individual's face, with the Morimoto et al. system, because such 
combination, would permit automatic machine-based face detection and recognition in 
uncontrolled environments (Page 1, chapter 1, line 11-12). 

(b) Obviousness in view of Watanabe 

Watanabe, in analogous environment, teaches a surveillance system, 
surveillance method and surveillance program, where using the multiple tables (tables 
of Fig. 9 and 10) of image classification events (tables of Fig. 9 and 10 are read as 
multiple tables), occasions, locations, or place (Fig. 9, paragraph [0088], Iine1-13, and 
paragraph [0089], line 1-3). 

It is desirable to provide a surveillance system whereby specific persons can be 
detected readily from visiting persons, without placing a large burden on the 
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surveillance operator. The Watanabe's teaching where using multiple tables is to 
achieve this goal. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time of the invention, to apply the Watanabe's system, where using 
multiple tables, with the combination Morimoto et al. and Tony J., because such feature 
provides a surveillance system whereby specific persons can be detected readily from 
visiting persons, without placing a large burden on the surveillance operator ((see the 
Abstract, line 1-4). 

(2) Regarding claims 6 and 25: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the image data component further comprises a set of database 
(attribute data) links to the tables of image classification categories (column 4, line 64- 
65), (the set of database is read as the attribute data set to the image data). 

8. Claims 11, 31-32, and 41-42 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Morimoto et al. (US 6,418,235) in view of Tony S. Jebara "3D Pose 
Estimation and Normalization for Face Recognition", Department of Electrical 
Engineering, McGill University, May 1996, chapter 4, P-P 61-68, and Jonas (US- 
PGPUB 2004/0210763). 

(1) Regarding claims 11, 31, and 41: 

As shown in figure 4, Morimoto et al. disclose one or more processor-readable 
media having stored thereon processor-readable code (column 3, line 20-22) including 
a database of face print data corresponding to detected face regions within images 
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acquired (column 1, line 10-11) with an image acquisition device (13 in figure 1, column 
3, line 4) and digitally-stored (21 in figure 2,column 3, line 24-25) wherein the face print 
image data are stored within the media for access by processor (column 3, line 23-25) 
comprises: 

(a) an image data component (extracted means) including acquired digital image 
data including content data and unique identifiers corresponding to individual acquired 
digital images or face regions therein, or both; (column 1 , line 47-49); 

(b) an identity data component (storage means) including an identification listing 
of known identities to which identified face regions detected within the acquired digital 
image data have been determined to correspond; (column 1, line 45-47); 

(c) a face recognition data component (collation means), comprising for an 
individual known identity (column 1, line 49-54): 

(i) an appearance table (table of figure 3) including one or more identity 
entries for the known identity, (column 3, line 59-61), (the table of figure 3, 
comprises an appearance characteristics corresponding to the identities entries); 

(ii) one or more identity tables corresponding to the one or more identity 
entries (given name) in the appearance table (column 3, line 59-60); 

(iii) one or more face class tables corresponding to one or more face class 
entries of the one or more identity tables, where each face class table comprises 
one or more face print image entries corresponding to face prints (column 3, line 
47-48; and column 3, line 56-59). 
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However, Morimoto et al. do not explicitly mention the following items: 

1) the normalizing of face regions, comprising spatially normalized face region, 
which are normalized with respect to size, orientation or pose, or luminance normalized 
face region, or combination thereof. 

2) wherein the system further comprises a set of links to a relationship list or a 
group membership list or both. 

(a) Obviousness in view of Tony J. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions comprises spatially normalized face 
region, which are normalized with respect to the pose of an individual's face (Page 61, 
line 17-22), size (average 3D face) (Fig. 4.7, Page 65, line 9-13), and luminance 
normalized face region (page 61, line 9-10). 

It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where normalizing face with respect to poses of an individual's face, is to 
achieve this goal. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time of the invention, to apply the Tony's teaching of normalizing face with 
respect to pose of an individual's face, with the Morimoto et al. system, because such 
combination, would permit automatic machine-based face detection and recognition in 
uncontrolled environments (Page 1, chapter 1, line 11-12). 

(b) Obviousness in view of Jonas 

Jonas, in analogous environment, teaches a confidential data sharing and 
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anonymous entry resolution, where the system comprises a set of links (identifier) to a 
relationship list (paragraph [0039], line 1-3), (it is read that the identifier is linked to the 
relationship) 

It is desirable to provide a method for processing data in a database. The Jonas 
teaching, where the system comprises a set of links to a relationship is to achieve this 
goal. Therefore, it would have been obvious to one having ordinary skill in the art at the 
time of the invention, to apply the Jonas teaching, where the system comprises a set of 
links to a relationship, with the combination Morimoto et al. and Tony J., because such 
feature provides the processing of data in a database (paragraph [0012], line 1-2). 

(2) Regarding claims 32 and 42: 

Morimoto et al. further disclose one or more processor-readable media (column 
3, line 20-22), where the relationship list comprises data on relationships between the 
known identity and other identities named within the database, and wherein the group 
membership list comprises data on grouping of known identities based on family ties, 
hobbies, interests, group memberships, interpersonal relationships, or combinations 
thereof (Table of figure 3, column 3, line 52-55 and line 61-67). 

9. Claims 52-57 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Morimoto et al. and Tony J., as applied to claim 1 above, and further in view of 
Nicponski (US-PGPUB 2003/0128877). 
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(1) Regarding claims 52, 54, and 56: 

The combination Morimoto et al. and Tony J. teaches the parental claim 1. 
Furthermore, Morimoto et al. discloses one or more processor-readable media (column 
3, line 20-22) as described in claims 1,17, and 34 above. 

The combination Morimoto et al. and Tony S. do not teach explicitly the system, 
where the normalized face regions are normalized with respect to size. 

Nicponski, in analogous environment, teaches a method and system for 
processing images for themed imaging services, where the normalized face regions are 
normalized (paragraph [0123], line 1-3) with respect to size (paragraph [0123], line 1). 

It is desirable to process a digital image to assign useful meaning to human 
understandable objects and then utilize the results obtained in the further processing of 
the digital image. The Nicponski's teaching, where normalizing face regions with respect 
to the size, is to achieve this goal. Therefore, it would have been obvious to one having 
ordinary skill in the art at the time of the invention, to apply the Nicponski's teaching, 
where normalizing face regions with respect to the size, with the combination Morimoto 
et al. and Tony J., because such feature processes a digital image to recognize and 
thereby assign useful meaning to human understandable objects, attributes or 
conditions and then to utilize the results obtained in the further processing of the digital 
image (paragraph [0183], line 4-7). 



Application/Control Number: 10/764,335 Page 26 

Art Unit: 2624 

(2) Regarding claims 53, 55, and 57: 

The combination Morimoto et al. and Tony J. teaches the parental claims 52, 54 
and 56. Furthermore, Morimoto et al. disclose one or more processor-readable media 
(column 3, line 20-22) as described in claims 52, 54 and 56 above. 

However, the combination Morimoto et al. and Tony J. Morimoto et al. do not 
explicitly mention the system, where the normalized face regions are normalized with 
respect to a standard size based on separation of eyes, nose or mouth, or combination 
thereof. 

Nicponski, in analogous environment, teaches a method and system for 
processing images for themed imaging services, where the normalized face regions are 
normalized (paragraph [0123], line 1-3) with respect to standard size (paragraph [0123], 
line 1) based on separation of eyes, nose, or mouth (paragraph [0125], line 1-3), (the 
sub-region size which contains facial features is read as the same concept as the 
separation of eyes, nose, or mouth). 

It is desirable to process a digital image to assign useful meaning to human 
understandable objects and then utilize the results obtained in the further processing of 
the digital image. The Nicponski's teaching, where normalizing face regions based on 
separation of eyes, nose, or mouth, is to achieve this goal. Therefore, it would have 
been obvious to one having ordinary skill in the art at the time of the invention, to apply 
the Nicponski's teaching, where normalizing face regions based on separation of eyes, 
nose, or mouth, with the combination Morimoto et al. and Tony J., because such feature 
processes a digital image to recognize and thereby assign useful meaning to human 
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understandable objects, attributes or conditions and then to utilize the results obtained 
in the further processing of the digital image (paragraph [0183], line 4-7). 

10. Claims 14 and 44 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Morimoto et al. and Tony J., as applied to claim 1 above, and further in view of Lee 
etal. (7,092,555). 

The combination Morimoto et al. and Tony J. teaches the parental claims 13 and 
43 above. 

However, the combination Morimoto et al. and Tony S. do not teaches explicitly 
the one or more processor-readable media, where the appearance characteristic is 
distinguished as determined from a sufficient difference in value of at least one face 
classifier parameter between faceprints and associated normalized face region 
determined to correspond to the same known identity, or based on user input, or both. 

Lee et al., in analogous environment, teaches a system for registering and 
authenticating human face, where the appearance characteristic is distinguished 
(column 6, line 13-17) as determined from a sufficient difference in value of at least one 
face classifier parameter (column 4, line 62-63) between faceprints and associated 
normalized face region (column 4, line 57-61) determined to correspond to the same 
known identity (column 8, line 24-30). 

It is desirable to perform reduced face authentication using the optimal set of readily 
distinguishable features at a training step. The Lee et al. teaching, where distinguishing 
the appearance characteristic is to achieve this goal. Therefore, it would have been 
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obvious to one having ordinary skill in the art at the time of the invention, to apply the 
Lee et al. teaching, where distinguishing the appearance characteristic, with the 
combination Morimoto et al. and Tony J., because such feature performs reduced face 
authentication using the optimal set of readily distinguishable features at a training step 
(column 9, line 22-28). 

11. Claims 18-19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Morimoto et al. (US 6,418,235) in view of Tony S. Jebara "3D Pose Estimation and 
Normalization for Face Recognition", Department of Electrical Engineering, McGill 
University, May 1996, chapter 4, P-P 61-68, and DeLuca et al. (US-PGPUB 
2004/0223063). 

(1) Regarding claim 18: 

As shown in figure 4, Morimoto et al. disclose one or more processor-readable 
media having stored thereon processor-readable code (column 3, line 20-22) including 
a database of face print data corresponding to detected face regions within images 
acquired (column 1, line 10-11) with an image acquisition device (13 in figure 1, column 
3, line 4) and digitally-stored (21 in figure 2,column 3, line 24-25) wherein the face print 
image data are stored within the media for access by processor (column 3, line 23-25) 
comprises: 

(a) an image data component (extracted means) including acquired digital image, 
or a pointer to the location of said image (column 1, line 47-49), and additional data 
(attribute data) associated with said image (column 1, line 48), including content data 
(face data) (column 3, line 50-52), and unique identifiers (personal data) (column 3, line 
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44-46), (the personal data is read as unique identifier, such as a given name of person 
to be registered and registration item) corresponding to the acquired digital images or 
face regions therein, or both, and wherein the image data component further comprises 
an image list of the acquired digital image data (attribute data) (column 4, line 13-17); 

(b) an identity data component (storage means) including an identification listing 
of known identities to which identified face regions detected within the acquired digital 
image data have been determined to correspond (column 1, line 45-47); 

(c) a face recognition data component (collation means), comprising for an 
individual known identity (column 1, line 49-54): 

(i) one or more identity tables (table of figure 3) corresponding to one or 
more identity entries (column 3, line 3, line 48-50), (the table of figure 3 has the 
identity characteristics corresponding to the identity entries). 

(ii) one or more face class tables (table of figure 3) corresponding to one 
or more face class entries of the one or more identity tables, wherein each face 
class table comprises one or more faceprint entries (Column 3, line 56-67), (the 
table of figure 3 has the face class table corresponding to face class entries of 
the identity table). 

However, Morimoto et al. do not explicitly mention the following items: 

1) normalizing of face regions, comprising spatially normalized face region, which 

are normalized with respect to size, orientation or pose, or luminance normalized face 

region, or combination thereof. 
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2) wherein the one or more groups of image data further include image metada 
including anthropometrical information. 

(a) Obviousness in view of Tony J. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions comprises spatially normalized face 
region, which are normalized with respect to the pose of an individual's face (Page 61, 
line 17-22), size (average 3D face) (Fig. 4.7, Page 65, line 9-13), and luminance 
normalized face region (page 61 , line 9-10). 

It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where normalizing face with respect to poses of an individual's face, is to 
achieve this goal. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time of the invention, to apply the Tony's teaching of normalizing face with 
respect to pose of an individual's face, with the Morimoto et al. system, because such 
combination, would permit automatic machine-based face detection and recognition in 
uncontrolled environments (Page 1, chapter 1, line 11-12). 

(b) Obviousness in view of DeLuca et al. 

DeLuca et al., in analogous environment, teaches a detecting red eye filter and 
apparatus using meta-data, where the image data image data include image metada 
(paragraph [0148], line 14-17) including anthropometrical information (paragraph [0069], 
line 4-5). 
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It is desirable to provide a good indication as to whether an object is an eye, 
based on analysis of other detected human objects in the image. The DeLuca et al. 
teaching, where the image data includes a metadata including anthropometrical 
information is to achieve this goal. Therefore, it would have been obvious to one having 
ordinary skill in the art at the time of the invention, to apply the DeLuca et al. teaching, 
where the image data includes a metadata including anthropometrical information, with 
the combination Morimoto et al. and Tony J., because such feature, can provide a good 
indication as to whether an object is an eye, based on analysis of other detected human 
objects in the image (paragraph [0017], line 4-6). 

(2) Regarding claim 19: 

As shown in figure 4, Morimoto et al. disclose one or more processor-readable 
media having stored thereon processor-readable code (column 3, line 20-22) including 
a database of face print data corresponding to detected face regions within images 
acquired (column 1, line 10-11) with an image acquisition device (13 in figure 1, column 
3, line 4) and digitally-stored (21 in figure 2,column 3, line 24-25) wherein the face print 
image data are stored within the media for access by processor (column 3, line 23-25) 
comprises: 

(a) an image data component (extracted mean) including acquired digital image, 
or a pointer to the location of said image (column 1, line 47-49), and additional data 
(attribute data) associated with said image (column 1, line 48), including content data 
(face data) (column 3, line 50-52), and unique identifiers (personal data) (column 3, line 
44-46), (the personal data is read as unique identifier, such as a given name of person 
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to be registered and registration item) corresponding to the acquired digital images or 
face regions therein, or both, and wherein the image data component further comprises 
an image list of the acquired digital image data (attribute data) (column 4, line 13-17); 

(b) an identity data component (storage means) including an identification listing 
of known identities to which identified face regions detected within the acquired digital 
image data have been determined to correspond (column 1, line 45-47); 

(c) a face recognition data component (collation means), comprising for an 
individual known identity (column 1, line 49-54): 

(i) one or more identity tables (table of figure 3) corresponding to one or 
more identity entries (column 3, line 3, line 48-50), (the table of figure 3 has the 
identity characteristics corresponding to the identity entries). 

(ii) one or more face class tables (table of figure 3) corresponding to one 
or more face class entries of the one or more identity tables, wherein each face 
class table comprises one or more faceprint entries (Column 3, line 56-67), (the 
table of figure 3 has the face class table corresponding to face class entries of 
the identity table). 

However, Morimoto et al. do not explicitly mention the following items: 

1) normalizing of face regions, comprising spatially normalized face region, which 
are normalized with respect to size, orientation or pose, or luminance normalized face 
region, or combination thereof. 

2) wherein the one or more groups of image data further include image metada 
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Including focusing distance of the lens at the time of acquisition, or effective digital 
camera sensor size, or both. 

(a) Obviousness in view of Tony J. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions comprises spatially normalized face 
region, which are normalized with respect to the pose of an individual's face (Page 61, 
line 17-22), size (average 3D face) (Fig. 4.7, Page 65, line 9-13), and luminance 
normalized face region (page 61 , line 9-10). 

It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where normalizing face with respect to poses of an individual's face, is to 
achieve this goal. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time of the invention, to apply the Tony's teaching of normalizing face with 
respect to pose of an individual's face, with the Morimoto et al. system, because such 
combination, would permit automatic machine-based face detection and recognition in 
uncontrolled environments (Page 1, chapter 1, line 11-12). 

(b) Obviousness in view ofDeLuca etal., 

DeLuca et al., in analogous environment, teaches a detecting red eye filter and 
apparatus using meta-data, where the image data image data include image metada 
(paragraph [0148], line 14-17) including focusing distance of the lens at the time of 
acquisition (paragraph [0030], line 1-3). 
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It is desirable to have the digital capture device that contains more data than 
traditional film based image. The DeLuca et al. teaching, where the image data image 
data include image metada is to achieve this goal. Therefore, it would have been 
obvious to one having ordinary skill in the art at the time of the invention, to apply the 
DeLuca et al. teaching, where the image data includes a metadata including focusing 
distance of the lens at the time of acquisition, with the combination Morimoto et al. and 
Tony J., because in such feature the digital capture device has an advantage that the 
image contains more data than traditional film based image has (paragraph [0010], line 
1-3). 

12. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Morimoto et al. (US 6,418,235) in view of Tony S. Jebara "3D Pose Estimation and 
Normalization for Face Recognition", Department of Electrical Engineering, McGill 
University, May 1996, chapter 4, P-P 61-68, and Enomoto (US-PGPUB 2003/0086134). 

As shown in figure 4, Morimoto et al. disclose one or more processor-readable 
media having stored thereon processor-readable code (column 3, line 20-22) including 
a database of face print data corresponding to detected face regions within images 
acquired (column 1, line 10-11) with an image acquisition device (13 in figure 1, column 
3, line 4) and digitally-stored (21 in figure 2,column 3, line 24-25) wherein the face print 
image data are stored within the media for access by processor (column 3, line 23-25) 
comprises: 
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(a) an image data component (extracted means) including acquired digital image, 
or a pointer to the location of said image (column 1, line 47-49), and additional data 
(attribute data) associated with said image (column 1, line 48), including content data 
(face data) (column 3, line 50-52), and unique identifiers (personal data) (column 3, line 
44-46), (the personal data is read as unique identifier, such as a given name of person 
to be registered and registration item) corresponding to the acquired digital images or 
face regions therein, or both, and wherein the image data component further comprises 
an image list of the acquired digital image data (attribute data) (column 4, line 13-17); 

(b) an identity data component (storage means) including an identification listing 
of known identities to which identified face regions detected within the acquired digital 
image data have been determined to correspond (column 1 , line 45-47); 

(c) a face recognition data component (collation means), comprising for an 
individual known identity (column 1, line 49-54): 

(i) one or more identity tables (table of figure 3) corresponding to one or 
more identity entries (column 3, line 3, line 48-50), (the table of figure 3 has the 
identity characteristics corresponding to the identity entries). 

(ii) one or more face class tables (table of figure 3) corresponding to one 
or more face class entries of the one or more identity tables, wherein each face 
class table comprises one or more faceprint entries (Column 3, line 56-67), (the 
table of figure 3 has the face class table corresponding to face class entries of 
the identity table). 

However, Morimoto et al. do not explicitly mention the following items: 
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1) normalizing of face regions, comprising spatially normalized face region, which 
are normalized with respect to size, orientation or pose, or luminance normalized face 
region, or combination thereof. 

2) wherein the image data component further includes additional image data 
associated with circumstances of acquisition of a parent image and associated face 
region corresponding to a group of image data. 

(a) Obviousness in view of Tony J. 

Tony J., in analogous environment, teaches a Face normalization and 
recognition, where the normalized face regions comprises spatially normalized face 
region, which are normalized with respect to the pose of an individual's face (Page 61, 
line 17-22), size (average 3D face) (Fig. 4.7, Page 65, line 9-13), and luminance 
normalized face region (page 61, line 9-10). 

It is desirable to develop a vision system which would permit automatic machine- 
based face detection and recognition in uncontrolled environments. The Tony's 
teaching, where normalizing face with respect to poses of an individual's face, is to 
achieve this goal. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time of the invention, to apply the Tony's teaching of normalizing face with 
respect to pose of an individual's face, with the Morimoto et al. system, because such 
combination, would permit automatic machine-based face detection and recognition in 
uncontrolled environments (Page 1, chapter 1, line 11-12). 



Application/Control Number: 10/764,335 Page 37 

Art Unit: 2624 

(b) Obviousness in view of Enomoto 

Enomoto, in analogous environment, teaches an apparatus and method for 
image processing, where the image data component further includes additional image 
data associated with circumstances of acquisition of a parent image (paragraph [0092], 
line 14-16) and associated face region corresponding to a group of image data 
(paragraph [0092], line 16-19). 

It is desirable to effectively executing red-eye color correction processing, freckle 
and wrinkle removing processing, trimming processing and the like for executing image 
processing based on image data. The Enomoto's teaching where the additional image 
data are associated with circumstances of acquisition of a parent image is to achieve 
this goal. Therefore, it would have been obvious to one having ordinary skill in the art at 
the time of the invention, to apply the Enomoto's teaching, where the additional image 
data are associated with circumstances of acquisition of a parent image, with the 
combination Morimoto et al. and Tony J., because such feature, effectively executing 
red-eye color correction processing, freckle and wrinkle removing processing, trimming 
processing and the like for executing image processing based on image data 
(paragraph [0008], line 1-8). 

13. Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Morimoto et al.,Tony J., and Enomoto, as applied to claim 20 above, and further in view 
of Okusa (US-PGPUB 2003/0158838). 

The combination Morimoto et al., Tony J., and Enomoto teaches the parental 
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claim 20. Furthermore, Enomoto discloses an image processing including red eye 
correction (Enomoto: paragraph [0008], line 4-5). 

However, the combination Morimoto et al., Tony J., and Enomoto do not explicitly 
mention that the circumstances comprising location of image acquisition, date and time 
of image acquisition, type of image acquisition device. 

Okusa, in analogous environment, teaches an image processing apparatus, 
comprising location of image acquisition (paragraph [0050], line 33-34), date (paragraph 
[0050], line 30-31) and time (paragraph [0050], line 17) of image acquisition, type of 
image acquisition device (paragraph [0050], line 4-5). 

It is desirable to eliminate the labor required to enter a keyword for retrieving 
image data, and that can reduce the time needed to prepare retrieval data. The Okusa's 
teaching, where comprising location of image acquisition, date and time of image 
acquisition, type of image acquisition device is to achieve this goal. Therefore, it would 
have been obvious to one having ordinary skill in the art at the time of the invention, to 
apply the Okusa's teaching, where comprising location of image acquisition, date and 
time of image acquisition, type of image acquisition device, with the combination 
Morimoto et al., Tony J., and Enomoto, because such feature eliminates the labor 
required to enter a keyword for retrieving image data, and that can reduce the time 
needed to prepare retrieval data (paragraph [0014], line 3-5). 
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Conclusion 

14. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 



Contact Information: 

15. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to AMARA ABDI whose telephone number is (571)270- 
1670. The examiner can normally be reached on Monday through Friday 8:00 Am to 
4:00 PM E.T.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wu Jingge can be reached on (571) 272-7429. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Jingge Wu/ 

Supervisory Patent Examiner, Art Unit 2624 



/Amara Abdi/ 
Examiner, Art Unit 2624 
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